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INTRODUCTION 


On September 22, 2017, the National Center for Special Education Research (NCSER) and the 
National Center for Education Research (NCER) at the Institute of Education Sciences (IES) 
convened a group of experts in policy, practice and research related to Career and Technical 
Education (CTE). The goal of the meeting was to seek input from researchers and practitioners 
in CTE to help highlight the key research questions that need to be addressed, the types of 
research that would best address these questions, the challenges faced in doing this work, and 
strategies for promoting research in CTE (see Appendix A for agenda). The discussion followed 
four major topics of inquiry: 


1. Who is served by CTE and who is left behind? 

2. What do we know—and need to know—about CTE policies, programs, and practices at 
the secondary and postsecondary levels? 

3. What are the critical methodological issues in CTE? 

4. What is needed to advance CTE research? 


The day was structured by these topics, and each session began with comments by selected 
TWG members, followed by a general discussion. The meeting concluded with each TWG 
member providing priority recommendations for IES. This report summarizes the key points and 
themes that emerged by topic." 


OVERVIEW OF CTE 


Staff from NCER and the National Center for Education Statistics (NCES) provided context for 
the day’s discussion with an overview of CTE programs and their funding sources, and national 
statistics on CTE student participation and outcomes. 


NCER Overview 

NCER staff member Corinne Alfeld provided an overview of CTE for context prior to the TWG 
discussion. CTE is a set of organized activities that develop the career and technical skills of 
secondary and postsecondary students who elect to enroll in CTE programs. CTE as part of the 
U.S. education system dates back to the Smith Hughes Act of 1917 and is currently funded 
through the Carl. D. Perkins Act of 2006 (Perkins IV).” CTE has changed substantially since it was 
known as “vocational education.” For example, Perkins IV required states to offer CTE programs 


* The use of “TWG members” throughout the summary can refer to two or more members and does not 
necessarily indicate consensus among all members. 

* The Perkins legislation defines CTE as “coherent and rigorous content aligned with challenging academic 
standards and relevant technical knowledge and skills needed to prepare for further education and careers in 
current or emerging professions,” and “learning that contributes to the academic knowledge, higher-order 
reasoning and problem-solving skills, work attitudes, general employability skills, technical skills, and occupation- 
specific skills, and knowledge of all aspects of an industry, including entrepreneurship, of an individual.” 


of study, and many have adopted some or all of the 16 career clusters,’ depending on 
alignment with their local and regional industries. 


Federal funding makes up a large part of the support for CTE programs. The Department of 
Education distributes Perkins funds via a formula to states, which in turn distribute funds to 
local education agencies and institutions of higher education.“ In addition to Perkins funds for 
secondary CTE, there are also Workforce Innovation and Opportunity Act (WIOA) funds 
available through the Department of Labor (DOL) and other DOL grants for specific training 
programs.” 


Perkins IV requires that recipients of CTE funding develop programs of study, which link 
secondary and postsecondary CTE programs in a rigorous sequence of courses beginning in high 
school and leading to a postsecondary degree or industry-recognized credential,° with dual 
enrollment options along the way.’ (The earlier iteration of this structure was called “Tech 
prep.”) This legislative requirement seems to have increased the number and quality of 
collaborations between CTE educators at the secondary and postsecondary levels regarding 
curriculum alignment. This alignment across levels is referred to as career pathways -- course 
sequences that span the secondary and postsecondary levels, with each successive level grade 
or year becoming more technically specialized. Several states have been heavily investing in 
career pathways. 


Approximately 85% of high school students have some CTE experience.® CTE programs at the 
secondary level are primarily through elective courses in comprehensive high schools. Some 
states have regional CTE centers that multiple districts share (and space/equipment is 
sometimes shared with local technical colleges). Career academies are becoming an 
increasingly popular way to offer CTE. These are small learning communities, with cohorts of 
students advancing together and learning both academics and CTE in a thematic way, for 
example in engineering, business, and the arts. Students in career academies, like students in 


3 https://careertech.org/career-clusters. A Career Cluster is a grouping of occupations and broad industries based 
on commonalities. The Association for Career and Technical Education and Advance CTE recognize the following 16 
career clusters: eAgriculture, Food & Natural Resources, eArchitecture & Construction, eArts, A/V Technology & 
Communications, «Business Management & Administration, eEducation & Training, «Finance, eGovernment & 
Public Administration, eHealth Science, eHospitality & Tourism, eHuman Services, «Information Technology, eLaw, 
Public Safety, Corrections & Security, eManufacturing, eMarketing, eScience, Technology, Engineering & 
Mathematics, eTransportation, Distribution & Logistics. 

It should be noted that NCES uses slightly different categories for analyses of its various datasets.. 

“In 2011-12, 37 states earmarked additional state funds for secondary CTE, but only 5 earmarked funds for 
postsecondary CTE. (U.S. Department of Education (2014). National Assessment of Career and Technical Education: 
Final Report to Congress. Washington, D.C.: Office of Planning, Evaluation and Policy Development, Policy and 
Program Studies Service.) 

° https://www.doleta.gov/wioa/ 

° Credentials administered by various industry groups, e.g. ASE for the automotive industry or CompTIA for the IT 
industry. 

7 Dual enrollment refers to college-level courses taken for college credit while a student in in high school. 

® National Center for Education Statistics (NCES) (2013). Trends in CTE Coursetaking. Data point (NCES 2014-901). 
Washington, DC: U.S. Department of Education. 


many traditional high schools, also have the opportunity to engage in work-based learning with 
local employers. There are approximately 8,000 academies across the country. At the 
postsecondary level, CTE instruction primarily takes place at community and technical colleges, 
where students can earn a certificate or an associate’s degree. There has been enormous 
growth in the quantity and quality of postsecondary CTE offerings, both for traditional-age and 
returning students. 


For the purposes of the TWG, NCER and NCSER are most concerned with CTE programs where 
measurable academic and technical skills outcomes are possible. These outcomes can be 
measured through course grades, technical skills assessments, industry certifications, 
certificates, and degrees. That said, NCER and NCSER staff are open to discussing a range of 
instructional practices, strategies, programs, and policies in career and technical education. 


NCES Overview 

NCES staff member Lisa Hudson then provided national CTE statistics collected by NCES” for 
context prior to the TWG discussion. Most high school data on CTE come from transcript 
studies attached to the National Assessment of Educational Progress (NAEP).”° Such studies are 
expensive and therefore take place only periodically. The last data collection was in 2009, and 
the next is planned for 2019. 


From past NAEP high school transcript data collections, we know that: 

e 85% of students earn at least one credit in CTE; 36% earn 3 or more CTE credits; 35% 
earn at least 2 credits in the same occupational area’’; 19% earn at least 3 credits in the 
same occupational area (these are called “concentrators”). 

e On average, high school students earn 2.5 occupational CTE credits. 

e Between 1990 and 2009, the average number of CTE credits earned decreased; at the 
same time, the average number of academic credits (non-CTE) earned increased. 

e On average, rural students take more CTE (3.2 credits) than urban (2.3 credits) or 
suburban (2.5 credits) students. 

e Males earn more CTE credits than females. Whites also earn more CTE credits than 
Hispanics, as do students with disabilities versus those without, and students who start 
high school with lower versus higher math achievement. 


From the 2007-08 Schools and Staffing Survey,” we know that: 
e 82% of all public high schools offer CTE (of these, 63% are regular high schools, 4% are 
CTE high schools, and 16% are other special focus high schools). 


Based on the Education Longitudinal Study of 2002,’° we know that: 


* https://nces.ed.gov/surveys/ctes/ 

*° https://nces.ed.gov/nationsreportcard/ 

** NCES often uses the word “occupational” in their coding of CTE courses. 
* https://nces.ed.gov/surveys/sass 

*? https://nces.ed.gov/surveys/els2002 


Half of secondary CTE concentrators (i.e., those with 3 credits or more) go on to earna 
postsecondary credential. 

Of 2004 graduates, those who earned more CTE credits in high school (particularly in the 
same occupational area) were less likely to earn a postsecondary credential 6 years later 
(2012) than students who took fewer CTE credits or who took CTE credits that were not 
in the same occupational area. 

Among students who earned a postsecondary credential, students who had earned 
more than 4 occupational CTE credits in high school (i.e., CTE concentrators with 3 
credits or more) were more likely to earn a postsecondary credential in an occupational 
field than those who had taken fewer occupational CTE credits. However, only 28% of 
high school CTE concentrators who earned a postsecondary credential earned that 
credential in the same occupational field in which they had concentrated. 

Among employed high school graduates without a postsecondary credential, those 
concentrators who had taken 4 or more occupational CTE credits in high school, and 
those concentrators who had taken at least 3 credits in the same occupational area, 
were more likely than those who had taken fewer occupational CTE credits to be 
employed in what they considered to be a “career path” job. 

Compared with baccalaureate students and subbacalaureate academic students, 
subbaccalaureate occupational students are more likely to be black, female, and older, 
and to come from families with lower levels of education. 


According to the Integrated Postsecondary Education Data System (IPEDS) ’ in 2015, we know 


that: 


Of all postsecondary credentials awarded in 2015, 38% were subbaccalaureate 
occupational credentials (23% certificates and 15% associate’s degrees). 


Based on data from the Beginning Postsecondary Students (BPS:2004/09)*° dataset we know 


that: 


Within 6 years after enrolling in postsecondary education, 56% of occupational 
subbaccalaureate students had completed a credential, as had 58% of academic 
subbaccalaurate students (no significant difference between the two rates). These rates 
were lower than the 79% completion rate of bachelor’s degree students. 

Among the 56 percent of students who completed a credential within 6 years of 
enrolling, completers of occupational bachelor’s degrees were more often employed 
than were completers of academic bachelor’s degrees (89% versus 86%). There were no 
significant differences in the employment rates between occupational and academic 
associate’s degree completers (86% versus 79%). (The estimate for the latter has a 
relatively large standard error, so this particular comparison may be unreliable.) 

Those who completed occupational certificates were employed at a lower rate (78%) 
than those who completed other occupational credentials (occupational associate’s 


* https://nces.ed.gov/ipeds 
* https://nces.ed.gov/surveys/bps 


degree 86%; academic associate’s degree 79%; occupational bachelor’s degree 89%; 
academic bachelor’s degree 83%). 


WHO IS SERVED BY CTE, AND WHOIS LEFT BEHIND? 


Following the presentations by NCER and NCES staff, the TWG members considered the CTE 
experiences of students from different economic and demographic backgrounds and students 
with disabilities, as well as the differing needs of CTE students at the secondary and 
postsecondary levels. 


Some research has been conducted to date on the benefits of CTE course-taking on education 
and employment outcomes. For example, studies have shown that concentrators (students 
who earn 3 or more credits in a single CTE program of study) are more likely to graduate, enroll 
in college, be employed, and earn higher wages than similar students who did not concentrate 
in CTE, as noted in the NCER and NCES staff presentations.’° CTE concentrator enrollment data 
are often disaggregated only by career cluster. However, more research is needed to 
understand the CTE experiences of students from disparate economic and demographic 
backgrounds and of students with disabilities. 


The National Longitudinal Transition Study-2 (NLTS2) is a nationally representative and 
longitudinal sample of students with disabilities in high school.’’ An analysis was conducted 
assessing demographic differences in CTE course-taking (by gender, race/ethnicity, or 
household income) for these students as compared to those without disabilities. ® Differences 
were observed by gender and race/ethnicity among students without disabilities. In contrast, 
no demographic differences were observed among students with disabilities; although 
significant differences were observed by disability category (e.g., Individuals with Disabilities 
Education Act*® categories) and for those with multiple disabilities. In addition, high school 
graduates with disabilities were more frequently CTE concentrators than were those with no 
reported disabilities. Another analysis of the NLTS2 data for students with emotional 
disturbances (ED) as well as for those with learning disabilities (LD) found that participation in 
CTE is positively associated with these postsecondary outcomes up to two years after leaving 
high school (Wagner et al., 2015; 2017).”° ** 


TWG members noted that research is lacking on specific subpopulations in CTE. For example, 
there has been little research on the barriers faced by female and male students in programs in 
which they have traditionally been underrepresented (e.g., male participation in health fields 


https://edexcellence.net/publications/career-and-technical-education-in-high-school-does-it-improve-student- 


outcomes 

” https://ies.ed.gov/ncser/projects/nits2/ 

* https://ies.ed.gov/ncser/pubs/20123000/pdf/20123000.pdf 

* https://sites.ed.gov/idea/ 

*° http://journals.sagepub.com/doi/abs/10.1177/216514341562639 
** http://journals.sagepub.com/doi/abs/10.1177/0022219415574774 


and female participation in STEM fields). Furthermore, when data are disaggregated, the 
sample sizes are often not large enough to reach meaningful and useful conclusions about, for 
example, student participation in CTE by disability status or demographic subgroup. Such data 
are necessary to be able to identify effective practices that support diverse student populations 
and to understand and shift trends observed in over- or under-representation of certain 
students in CTE programs (e.g., that may be reinforcing stereotypes about who participates in 
CTE). Research is needed to help understand the CTE experiences for the range of students 
currently served in order to see if and where changes are needed (e.g., appropriate supports for 
students with disabilities). 


Another concern for research on subpopulations is that CTE students may or may not self- 
report as belonging to a special population category.” The potential for underreporting may 
affect our understanding of who is participating in what programs and which programs lead to 
better education and economic outcomes for these students. Some TWG members suggested 
that CTE researchers should collaborate with state and local education agencies to identify 
effective ways to collect data on student outcomes, including for students with disabilities and 
those from disparate populations. 


The often limited access of special populations to career and technical education is another 
critical issue that needs investigation. For example, although representing a small percentage of 
the student population, high school students who are also parents are often transitioned to 
alternative school settings that do not offer CTE programs, which can reinforce economic 
disadvantage. At the postsecondary level, a lack of support systems such as child care, 
transportation, financial aid, and funds to purchase textbooks may prevent some students, 
including those returning to school after a prolonged absence, from being able to attend CTE 
programs. Research focused on the access to and support systems within CTE at both the 
secondary and postsecondary levels is needed. 


The National Technical Assistance Center on Transition (NTACT) has identified vocational 
(career and technical) education as a strong predictor of post-school outcomes for students 
with disabilities based on a review of existing studies. High school students with learning and 
intellectual disabilities who participate in CTE are more likely to have positive post-school 
outcomes. However, the literature about evidence-based practices is correlational in nature 
and does not disaggregate by disability, often because the subgroups are too small for 
statistical analyses. In addition, there are few experimental intervention studies, and those that 
exist often do not usefully describe the intervention for replication or use in practice. TWG 
members noted the importance of describing interventions going forward to identify the key 
components of interventions (e.g., curricula, strategies) that have potential for improving 
student outcomes. 


2 In the CTE field, the term “special populations” is often confused with “special education.” The Carl D. Perkins 
Career and Technical Education Act defines special populations as students who are economically disadvantaged, 
single parents, displaced homemakers, English language learners, students pursuing nontraditional careers, as well 
as students with disabilities. 


TWG members noted that at the postsecondary level, most students are focused on 
employment as the ultimate outcome. In contrast, at the secondary level, most students are 
exploring their career options, and therefore employment in the field is not necessarily an 
outcome that we should expect to correlate with the strength and quality of a CTE program. In 
fact, for high school students, learning what they don’t want to pursue as a career is an 
important exercise. A possible topic of research would be to look at the differences between 
career “explorers” and CTE concentrators, as well as the transition from explorer to 
concentrator. It would also be useful to include student voices — from various subpopulations, 
occupational areas, and education levels — to better understand their experiences and 
decisions. 


TWG members also raised concerns about the enduring stigma across the country of CTE as a 
mechanism to serve low-performing and minority students. TWG members cautioned that 
labeling and messaging influences not only perceptions about CTE but also what is taught to 
ensure student success over the long term. 


WHAT WE KNOW—AND WHAT WE NEED TO KNOW—ABOUT CTE 
POLICIES, PROGRAMS, AND PRACTICES 


During this session, the TWG members described new and promising trends in CTE, 
curricula/instructional practices that are in use but not supported by research, and what we 
need to know to better serve students from diverse backgrounds and students with disabilities. 
The session was divided into secondary and postsecondary discussions, although TWG 
members noted that this is not always a helpful distinction, as many programs serve a range of 
students and involve both secondary and postsecondary institutions. 


Secondary Education 

The discussion on secondary education focused heavily on industry-recognized credentials and 
on business and industry engagement in CTE. States are struggling with whether and how to 
incentivize industry-recognized credentials through their funding and accountability systems, 
and have concerns about the variability in the quality of credentials. Significant work remains to 
ensure that credentials align with industry requirements and to examine the impact of different 
types of credentials on student outcomes and wage trajectories. However, it is difficult for 
researchers to access students’ scores on these tests. In addition, increasing use of credentials 
raises concerns about equity of access to credentials for students with disabilities because 
many third-party credentialing tests are not subject to the accommodation requirements of 
IDEA. 


TWG members noted that business and industry have traditionally looked to postsecondary 
institutions as their immediate pipeline for workers. However, the case needs to be made to 


3 The National Occupational Competency Testing Institute (NOCTI) does provide and document accommodations 
on their tests, but this information is not currently publicly available. 


employers that there could be a return on their investment in the secondary environment as 
well. Business and industry can engage with and support secondary CTE in many ways, from 
serving as advisors, to offering internships and apprenticeships, to helping develop curricula. 
Research is needed to determine how companies are currently involved in CTE programs and 
what effect this involvement has on students. Research could inform the development of a 
model for successful partnership and to strengthen the role of work-based learning in high- 
school. 


Arkansas is an example of a state with strong support of CTE by the state legislature and with 
business and industry engagement in CTE. Within recent years, the state established a taskforce 
to identify high-value labor markets for students and instituted policy to ensure that public 
school offerings align with industry needs. In addition, the state broadened “seat-time” to 
include work-based learning, established career coaching programs starting in seventh grade, 
and passed legislation to provide CTE in Alternative Learning Environments” for adjudicated 
youth. Arkansas’s Career Training Institute serves as a bridge from secondary to postsecondary 
education for students with disabilities—providing them with training and credentialing and 
developing their “soft skills” in a peer environment. Business and industry are also leading 
efforts to improve the pipeline of skilled workers. For example, in one school district, the 
manufacturing industry partnered with the local Chamber of Commerce and developed a 
curriculum for a middle school program that now has a waitlist of those interested in 
participating. 


One emerging problem noted by TWG members is that the higher the perceived quality or 
prestige of the CTE program, the more exclusive it becomes. For example, many disadvantaged 
students who were once “dumped” into CTE classes are now unable to obtain access, given the 
increasing demand for the higher quality CTE programs among more advantaged students and 
their parents. In addition, some programs have increased expectations for students without 
providing additional supports that might be needed for students with disabilities to meet the 
new standards. This practice might have the unintended effect of limiting opportunities for 
students with disabilities. At the same time, there are some CTE programs that are more 
personalized than regular classes, and therefore students with disabilities do not need to 
request and/or receive accommodations. Research is needed to identify ways to overcome 
these access issues and what supports are needed for students that may struggle to succeed in 
CTE. 


TWG members also expressed concerns about inadequate CTE teacher training, particularly for 
experts who are recruited from industry without prior teacher preparation. Research is critically 
needed on pedagogical training for CTE teachers, recruiting and retaining teachers in CTE, and 
state policies on CTE teacher certification (including alternative certification). A general trend 


"In Arkansas, “Alternative learning environment" means “an alternate class or program within a public school or 
school district that affords all students an environment that seeks to eliminate barriers to learning for any student 
whose academic and social progress is negatively affected by the student's personal characteristics or situation.” 
http://www.arkansased.gov/divisions/learning-services/alternative-learning-environment 


was noted in the proliferation of CTE teacher training programs by for-profit and third-party 
organizations at the same time as the discontinuation of CTE teacher preparation programs at 
universities. TWG members added that many CTE educators are not adequately trained to 
recognize students with disabilities or to address their individual needs. 


TWG members highlighted additional topics of interest for future research to explore: (1) 
provision of CTE to younger students (e.g., beginning career exploration activities in middle 
school); (2) provision of CTE to all students (i.e., requiring all students to take a CTE course or 
courses before high school graduation or making all subjects career-focused, such as in a career 
academy); (3) effects of dual enrollment in CTE on postsecondary education and work 
outcomes (since many CTE programs now offer courses for college credit, due to the 2006 
Perkins requirement for programs of study); (4) rigorous integration of academic education and 
career-focused education (e.g. infusing higher quality mathematics, language arts, or science 
into CTE course content, which would potentially render the CTE course eligible to meet 
graduation requirements in an academic subject area); (5) effective classroom practices (e.g., 
use of performance assessments, links to real-world work environments, and other strategies 
to help students master CTE skills); and (6) implications of automation, technology, and artificial 
intelligence on the future of work. 


Postsecondary Education 

The discussion on postsecondary CTE focused on the need for institutional change primarily in 
community colleges to include a focus on career goals (with CTE a part of the conversation), 
standardization of course sequences and requirements needed to achieve true “stackability” 
(i.e. building on each other) of credentials, and on understanding the experiences of students 
with disabilities. TWG members cautioned against categorizing students as CTE or non-CTE, 
which they indicated reinforces the silo in which CTE tends to operate. The TWG members 
explained that the division of academic versus CTE breaks down at the postsecondary level. For 
example, students earning an associate’s degree (or other subbaccalaureate credential) in 
nursing are considered to be in “CTE,” whereas students obtaining a bachelor’s degree (ora 
postgraduate degree) in nursing are considered “academic,” when these are in fact just 
different “exit points” or credentials that can be obtained on the same career pathway. 


TWG members suggested that the field would be best served by framing the discussion about 
secondary to postsecondary pathways as a continuum that enables transparent and sequential 
transitions from secondary to 2-year and then to 4-year programs or to training or 
employment, with guidance for students to understand the sequences. Such guidance serves 
students best when it is delivered in-person by someone with expertise (i.e., meeting with 
coaches rather than navigating websites), particularly for students who face barriers, such as 
those who are first generation college students and/or have disabilities. Research should focus 
on how career pathway models that seamlessly connect levels can be scaled, with clear entry 
and exit points and stackable credentials. Unfortunately, many of the courses that lead to 
careers in manufacturing (e.g., welding, machining) tend to be non-credit courses at community 
colleges and, because non-credit courses are typically not counted on transcripts, it is difficult 
to obtain data for students in such programs. Research is needed to better understand how 


students experience transitions into and through CTE programs and what educational supports 
may be needed to help them progress successfully on a career pathway. As one TWG member 
stated, “You can’t have stackable credentials if students don’t know what stacks with what.” 


TWG members noted that, unfortunately, the division between CTE and academic courses 
remains a barrier to creating sequential pathways and stackable credentials. For example, 
students who complete a 2-year engineering technology program cannot easily transfer to a 4- 
year engineering program, largely because many lack the calculus coursework required by 
universities to award credit for the 2-year degree. As a result, TWG members questioned more 
generally why students do not receive academic credit for CTE courses, even when the course 
content includes rigorous academics (e.g., math, science). Research is needed on the academic 
rigor of CTE programs as well as how to more deliberately integrate academic content and 
requirements into CTE courses (at both the secondary and postsecondary levels). 


CRITICAL METHODOLOGICAL ISSUES IN CTE 


Except for one notable MDRC study on career academies in New York (2008)” and a recent 
study of CTE high schools in Massachusetts, ~° few causal studies (i.e., studies identifying the 
effect of CTE on student outcomes) on CTE have been conducted. In addition, literature on 
students with disabilities in CTE is generally not available. During this session, the TWG 
members discussed the need for more causal studies of CTE and the challenges in conducting 
these studies. In addition, they noted that research on CTE is needed including exploration, 
development, and measurement studies. Finally, the TWG members noted that there is a 
shortage of trained researchers, tools, and data to conduct this research. 


NTACT has identified a set of research-based instructional practices in secondary education for 
students with disabilities receiving special education in the classroom that would translate well 
into the CTE classroom. In support of state and local efforts to transition students with 
disabilities to postsecondary education and employment, NTACT evaluates and then 
categorizes practices and predictors (i.e., activities in school that correlate with post-school 
outcomes) based on high quality research studies (though few have been causal). 


Policymakers are increasingly focused on questions of program impact and they, along with 
those in the research community, want to understand the outcomes of student participation in 
CTE in different settings (e.g., high school, college settings) and via delivery models (e.g., single 
courses, career academies, integrated academic/CTE pathways). However, causal research in 
CTE is complicated by several of the following methodological issues as noted by the TWG 
members. 


Kemple, J & Willner, C.J. (2008). Career academies: Long-term impacts on labor market outcomes, educational 
attainment, and transitions to adulthood. MDRC. (Note: TWG members pointed out that, while the career 
academy study is important, the findings are not necessarily applicable to all CTE programs). 

= Dougherty, S.M. (2018). “The Effect of Career and Technical Education on Human Capital Accumulation: Causal 
Evidence from Massachusetts.” Education Finance & Policy, 13(2). 
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e The fact that students self-select into CTE makes it difficult to generalize their 
experiences to the larger student population. 

e In most cases, researchers cannot randomly assign students to CTE programs (although 
this challenge can be overcome when programs are oversubscribed). 

e Understanding and ensuring CTE program implementation fidelity is difficult, 
particularly when multiple components of programs (e.g., career academies) have not 
been clearly defined or studied. 

e CTE programs are often small and specialized. Enrolling the number of students needed 
in the sample for sufficient power can be challenging, particularly when estimating 
impacts for subgroups. 

e There is no standard measurement for participation in CTE (e.g., it may be based on 
credit hours, courses, or enrollment in a program or institution) making the assessment 
of participation challenging. 

e Identifying appropriate outcome measures is challenging especially when there are 
many and they can vary by program participation. 


Single-case design may provide a good starting point to study the experiences of CTE for 
students with disabilities. In addition, the development of state longitudinal data systems 
(SLDS) has created opportunities to perform more sophisticated descriptive analyses to 
understand patterns and systematic differences, for example, for girls, students with 
disabilities, students of color, and students from different socioeconomic backgrounds. With 
large datasets, there are opportunities to conduct rigorous research to evaluate outcomes of 
CTE-related policy changes, for example the outcomes associated with changes to teacher 
certification requirements in Wisconsin.”’ While randomized controlled trials (RCTs) are still the 
“gold standard,” they can be expensive and time-consuming. Researchers can use large 
datasets and employ propensity score matching (PSM) to create control groups in quasi- 
experimental designs. However, in such designs, there will always be unobserved variables 
(e.g., student motivation) that cannot be controlled for, and also unmeasured variables that can 
influence the results. 


TWG members also described the potential for analyzing data from states like Pennsylvania 
that include in their data systems the results of statewide assessments of CTE students’ end-of- 
course (EOC) technical skills. Unfortunately, the National Occupational Competency Testing 
Institute (NOCTI), which develops and administers these tests, cannot share the data with 
researchers because these data belong to the state.“® TWG members stated that the analysis of 
these data could help shed light on course taking and program design in relationship to student 
outcomes. These are the types of questions that could be studied if states had the resources 


7 TWG members discussed the use of the difference-in-difference method for addressing questions using these 
large datasets. However, this method would not meet What Works Clearinghouse (WWC) standards for causal 
studies. 

*8 It was also noted that NOCTI collects data on which students had accommodations for their tests and that they 
also administer the Manufacturing Skills Standards Council (MSSC) test and the potential for research using these 
data. 
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(i.e., funding and expertise) to analyze EOC CTE test data. Similarly, industries that administer 
technical competency tests typically do not share data with researchers; these test scores 
would be another “gold mine” of data if they could be accessed in some way. Finally, in states 
like Texas, Arkansas, Ohio, and Florida that have now connected education and employment 
data that can be examined longitudinally, the value and impacts of industry-recognized 
credentials in the marketplace can also be studied. 


Of course, rigorous research in CTE cannot happen without a pool of trained CTE researchers. 
Unfortunately, this pool appears to be shrinking for several reasons, according to the TWG 
members. First, the number of university training programs across the country is declining as 
CTE faculty (who conduct CTE research and train new researchers) have not been replaced 
when they leave their university positions. Second, CTE research, like CTE itself, is stigmatized. 
There is a need to change attitudes by changing language and labels to make CTE more 
attractive to researchers from other fields (e.g., sociology, economics, psychology, counseling, 
special education). 


TWG members concluded that the field needs to re-conceptualize CTE — including better 
defining CTE students, instructors, programs, and measures — and identify the critical research 
questions in order to encourage more research in this field. 


WHAT IS NEEDED TO ADVANCE CTE RESEARCH? 


During this session TWG members discussed ways to advance CTE research, including greater 
collaboration among researchers and between researchers and practitioners, support of the 
next generation of CTE researchers, and the development of strategies to make CTE research 
more useful for policy and practice. State CTE administrators want to know how to identify 
quality CTE programs so they know how to spend their dollars most effectively on programs 
that best meet the needs of students. Policymakers also want to know what “works” and what 
the benefits are of such investments. The field also needs to disaggregate data to understand 
what works “for whom” as well as “under what conditions,” as the case was made throughout 
the meeting that little is known about student subgroups’ participation in CTE. 


The TWG members urged IES to continue to support and promote CTE research as a special 
topic in its Requests for Applications (RFAs). Placing a greater focus on CTE in IES funding 
opportunities would foster the interest of researchers to propose exploration, development 
and efficacy studies on a variety of topics in CTE, including work-based learning. Researchers 
interested in special education and secondary transition could develop and test CTE 
interventions that can improve transition outcomes for students with disabilities. It was noted 
that CTE is already a special topic under NCER research grants but currently is not for NCSER 
grants (such research would fit in the Transition Outcomes for Secondary Students with 
Disabilities topic in NCSER). The TWG members encouraged IES to lead efforts to advance CTE 
research by spearheading a Call to Action for other disciplines to engage in CTE research (e.g., 
learning science, business, social science). 
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To apply for and obtain IES grants to conduct CTE research, CTE experts would benefit from 
workshops on research design and from collaborating with methodologists who understand 
CTE or with related expertise. CTE practitioners, policymakers, and researchers could also 
propose research concepts to pitch to methodologists that may be focused on research in 
related areas. In addition, more scholarly research that is grounded in a sound theoretical 
framework (and publishable in a peer-reviewed journal) is needed when undertaking CTE 
studies to continue to build this body of research. 


The TWG members also highlighted the importance of collaborative cross-institutional and 
cross-agency efforts to advance CTE research and recommended the following for IES: 


Promote collaboration between researchers and state-level personnel to examine 
extant datasets and assess student outcomes in CTE, perhaps through partnership 
grants or Goal 1 exploration projects.*° 

Encourage cross-agency partnerships within states, and encourage states to think 
concretely about their administrative databases and how they can be useful in research 
to answer their questions and inform practice in CTE. 

Help states to improve access to their data (e.g., on coursetaking, credentials, 
demographics, delivery models, programs, practices) by researchers. 

Support the institution of a federal and state data-sharing agreement. 

Collaborate with the Office of Special Education and Rehabilitation Services (OSERS) to 
inform research-based practices, assessment, implementation, and iterative 
development and evaluation focused on the needs of individuals with disabilities in CTE 
(secondary and post-secondary). 

Collaborate with other federal agencies, such as the Department of Labor, that fund 
similar research to coordinate the advancement of research on CTE. 

Identify strategies to support collaboration with national research centers that are 
already working with states around CTE (e.g., Arkansas career coaching program). 


Other TWG member recommendations to IES for the promotion of research in CTE included the 
following. 


Fund studies that are large enough to disaggregate by CTE field of study, content, and 
student subgroup (Including disability and type of disability). 

Award small grants to doctoral students to conduct CTE research. 

Fund research on ways to measure CTE outcomes. 

Consider that reviewers may be biased against CTE as a topic for grant funding due to its 
stigma. 


TWG members also identified the following strategies for advancing CTE research and practice 
in general. 


* It was suggested that Advance CTE (association of CTE state directors) can encourage this. 
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e Communicate about CTE in a way that recognizes it encompasses many fields, rather 
than as a single school subject. 

e Encourage professors who have opportunities to foster interest in CTE research among 
their graduate students, and support the next generation of researchers and teachers 
(e.g., through fellowships, grants). 

e Improve research literacy among CTE practitioners to bridge the gap between research 
and practice. 

e Identify overlap with other fields and encourage collaborations focused on CTE. 


Finally, TWG members encouraged studies that produce concrete data on CTE student 
outcomes and skills across the continuum of high school and college (not just K-12) and beyond, 
and studies that examine the educational benefits of particular instructional approaches rather 
than focusing solely on whether a CTE program “works” or “does not work” in any given study. 


LIGHTNING ROUND: WHAT IS EACH PARTICIPANT’S TOP 
RECOMMENDATION FOR IES? 


TWG members shared their top recommendations for IES related to CTE research needs, 
funding, and strategy. These included points made by TWG members throughout the meeting 
as well as additional points that the TWG members felt should be included as top 
recommendations for IES and the CTE research field. Their comments are presented as follows 
and in no particular order. 


e Promote the study of teacher characteristics, along with teacher preparation programs, 
to better understand what makes a quality CTE teacher as well as how best to increase 
the pipeline of teachers, especially those from industry. 

e Identify the programs that work best for students and that are scalable, feasible, and 
fundable. 

e Encourage access to and evaluations of data on CTE in state administrative databases 
(and identify what other information should be collected) and links to agency databases 
wherever possible. 

e Support research on the study of dual enrollment as a strategy to increase college entry, 
persistence, and completion. 

e Collaborate to understand and disseminate how CTE and special education programs 
work together to address barriers for students with disabilities. 

e Encourage the examination of the settings and relationships needed to facilitate 
learning and the transition to adulthood in general for students (which is a larger 
question but that can also involve CTE). 

e Promote the examination of the impact of CTE on vulnerable student populations (i.e., 
those with multiple disadvantages). 

e Support the study of effective pedagogy and practices that increase participation of 
underrepresented students in CTE and nontraditional CTE, particularly around STEM and 
workforce diversity. 
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Encourage the conduct of descriptive research on transition points, that is, factors that 
influence positive transitions throughout the pathway linking secondary to 
postsecondary school, work, and life and the choices to enter various career pathways. 
Engage industry representatives in conversations about research on the quality and 
impact of their standards and credentials and how well CTE programs use them and 
align with their workforce objectives. 

Support research on academic integration without sacrificing the occupational content 
that is taught in CTE settings. 

Synthesize and disseminate the major questions and issues in the CTE field and follow a 
comprehensive approach to research that helps all students (not just CTE students) at all 
levels of the pathway in secondary to postsecondary life to choose programs and 
careers. 


In addition to the recommendations made earlier in the meeting, TWG members in the 
lightning round specifically suggested that IES consider the following when developing funding 
opportunities for research. 


Include CTE as a focus of research in all IES funding announcements. This may spark 
ideas and new partnerships. 

Consider adding CTE as an assigned topic area when soliciting proposals for Regional 
Education Labs (RELs).°° 

Be specific about what IES is looking for regarding CTE research (as informed by the field 
and driven by the needs of the field). 

Require that special education, CTE, and vocational rehabilitation inform each other and 
collaborate together to coordinate and plan for the conduct of needed research. 


LESSONS FROM THE DAY 


Commissioners Tom Brock and Joan McLaughlin summarized the lessons of the day regarding 
research in CTE. Based on the day’s discussions, potential areas for IES consideration include 
the following: 


With the wide variety of CTE programs, more descriptive work on programs and 
policies, and how they are being implemented, is needed. 

We need to better understand the issues around access to CTE. Although CTE as a 
field has been criticized for tracking and holding back students, there is another side 
to the story; many students who would benefit from participating in CTE are not 
gaining access to or enrolling in CTE programs. 

The transition from high school to postsecondary education should be more fluid 
and connected. An area for examination is how to bridge the divide between high 
school and college CTE. 


°° The REL program is administered by the National Center for Educational Evaluation and Regional Assistance: 


https://ies.ed.gov/ncee/. 
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As IES develops its research and funding priorities for the CTE field, it should 
consider a broad range of issues, including the need for more exploratory research, 
the use of state and industry datasets, development of curricula and assessments, 
and efficacy and effectiveness of teacher and researcher training. 
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Appendix A 


Gather in meeting room 
Welcome (Joan McLaughlin, NCSER; Tom Brock, NCER) 


Overview of CTE 
e What do we know about CTE programs? 
e How are CTE programs funded? 
e What do the national statistics say about who takes CTE? 


Who is served by CTE and who is left behind? 
e What are the experiences in CTE for students from different economic and 
demographic backgrounds? 
e What are the experiences in CTE for students with disabilities (or those without 
a diagnosis but facing challenges)? 
e How do the needs of secondary and postsecondary CTE students differ? 


BREAK 


What do we know — and need to know — about CTE policies, programs, and practices? 
(Secondary discussion, followed by Postsecondary discussion, of these questions) 
e What are new and promising trends in CTE? 
e = What curricula/instructional practices are in widespread use but have little 
support from research? 
e What do we need to know to better serve students with disabilities and those 
from different demographic backgrounds? 


LUNCH (on-site) 


What are the critical methodological issues in CTE? 
e Why are there so few causal studies? How do we encourage more of this work? 
e What kinds of exploration and development work are still needed? 
e What do we need to learn about CTE program implementation? 
e What outcomes matter for students in CTE? Do we have the right tools and data 
sources to measure these outcomes? 


BREAK 


What is needed to advance CTE research? 
e How can we encourage greater collaboration among researchers and between 
researchers and practitioners? 
e How can we support the next generation of CTE researchers? (e.g., graduate 
students, early career researchers, researchers from other fields) 
e What strategies will make CTE research more useful for policymaking and 
practice? 


Lightning Round: What is each participant’s top recommendation for IES? 
Summarize lessons from the day and next steps (Tom Brock, Joan McLaughlin) 


ADJOURN 
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